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Abstract. Interactive Scores is a formalism for the design and performance of interactive 
scenarios that provides temporal relations (TRs) among the objects of the scenario. We 
can model TRs among objects in Time Stream Petri nets, but it is difficult to represent 
global constraints. This can be done explicitly in the Non-deterministic Timed Concurrent 
Constraint (ntcc) calculus. We want to formalize a heterogeneous system that controls in 
one subsystem the concurrent execution of the objects using ntcc, and audio and video 
processing in the other. We also plan to develop an automatic verifier for ntcc. 


Introduction 

Interactive Scores (IS) are currently used for the design and performance of Electroa¬ 
coustic music lAllOSal and live spectacles [ Bal09 ] (e.g., interactive theater plays and inter¬ 
active museums). Both applications are based on Petri nets |Allf)8b] . The main purpose of 
IS is to provide temporal relations; for instance, precedence between two objects and rela¬ 
tions between their durations. Recently, we extended IS to support conditional branching 
together with temporal relations [TBIOj . It is now possible to represent loops and choices. 

We can model temporal relations in Time Stream Petri nets (TSPN) |Sen95] . but it is 
difficult to represent global constraints involving (possibly) all the objects of the scenario. 
Instead, in Concurrent Constraint Programming (cep) }Sar92] there are agents that reason 
about partial information contained in a constraint store ; thus, global constraints are inher¬ 
ent in ccp. However, there is not discrete time in ccp, which makes it difficult to represent 
reactive systems. 

There are some IS models based on extensions of ccp with discrete time. An example 
is the Non-deterministic Timed Concurrent Constraint (ntcc) model of IS |Nie021 IAll06| . 
Ntcc is an extension of ccp for non-determinism, asynchrony and discrete time. 

Ntcc belongs to a family of formalisms called process calculi. Process calculi has 
been applied to the modeling of interactive multimedia systems [Alllll ITorl4al ITB091 
IQlalll ITorl2al ITor09al ITorlOal ITorl5al IAra09l ITorl2bl ITor09bl ITorlObl ITorlOcl ITB08] 
and spatially-explicit ecological systems |Phil3al ITorl4bl IPhil3bl ITorl5b| . 

In the declarative view, ntcc processes can be interpreted as linear temporal logic 
formulae |Pnu77| . The ntcc model includes an inference system in this logic to verify 
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properties of ntcc models. This inference procedure was proved to be of exponential time 
complexity. Nevertheless, we believe practical automatic verification could be envisioned 
for useful subsets of ntcc via model checking (see [Fal06] h At present, there is no such 
automatic verifier for ntcc. 

Automated verification for ntcc will provide information about the correctness of the 
system to computer scientists, and will provide important properties about the scenario to 
its designers and users; for instance, reachability and liveness. We plan to augment ntcc 
models of IS with these features. 

1. Current and future work 

Functional AUdio STream (Faust) [Qrl04] is a programming language for signal pro¬ 
cessing with formal semantics and Ntccrt [TB09j is a real-time capable interpreter for ntcc. 
We implemented a signal processing prototype where Faust and Ntccrt interact together. 
In the future, we want to define formal semantics to describe a heterogeneous system that 
includes three subsystems: (i) one based on ntcc to control discrete events from the user 
and to synchronize the objects of the scenario, (ii) another one based on Faust to process 
audio and video, and finally (iii) one in charge to load and play audio and video files. 

At the time of this writing, there are no formal semantics of a heterogeneous system that 
synchronizes concurrent objects, handles global constraints, and controls audio and video 
streams. Modeling this kind of systems will be useful in other domains such as machine 
musical improvisation [Ass04| and music video games. An advantage over the existing 
implementations of these systems will be verification. 

In the proof system of ntcc, we can prove properties like “10 time units (TUs) after 
the event eg, during the next 4 TUs, the stream B is the result of applying a gain filter 
to the stream A”. However, real-time audio processing cannot be implemented in Ntccrt 
because it requires to simulate 44100 TUs per second to process a 44.1 kHz sound. If we 
replace some ntcc processes by Faust plugins, we can execute such system efficiently, but 
we cannot verify that the properties of the system hold. 

There are two open issues: (i) how to prove that a Faust plugin that replaces a ntcc 
process respect the temporal properties proved for the process, and (ii) whether an im¬ 
plementation of Interactive Scores in Ntccrt can be as efficient as the existing Petri nets 
implementation, or as one using synchronous languages such as Signal [Gau87j . although 
the performance results from Ntccrt are promising^. 
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l\Ve ran a prototype of a score with conditional branching in Ntccrt. The score contains 500 temporal 
objects. The average duration of each time-unit was 30 ms, which is compatible with real-time interaction. 
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